Camera Position Reconstruction and Tight Direction Networks

A concrete reconstruction problem arising in Computer Vision motivates our investigation of combinatorial variations on the problem of drawing a graph with given direction vectors associated to its edges. We formulate solutions in dimension 2 and report on experimental results done with simple implementations. Direction Networks. A (planar)direction network is a graph G = (V,E) with slopes (or direction vectors) associated to its edges. A compatible embedding is given by coordinates for the vertices V inducing edge directions equal to the given ones. A direction network is tight if it has a unique embedding, up to translations and scalings, it is inconsistent if it has no embedding at all, and it is loose or under constrained if it has more than one (non similar) embedding. An inconsistent network is usually over-constrained: some of the edge directions can be inferred algebraically from other edge directions, and thus a small error in the data leads to inconsistency. In the under-constrained case, the space of embeddings is a linear subspace whose dimension measures how loose the network is. See Fig. 1. Fig. 1. Left: a loose graph and two non-similar parallel redrawings. Right: an overconstrained case, with consistent and inconsistent directions (the rightmost case has no solution for the given direction of the hanging edge.) The Camera Position Reconstruction Problem. Our motivating problem comes from the MIT City Project [1]. A robot moves around the MIT campus taking pictures of the buildings. It seeks to reconstruct a model of the 3dimensional urban environment from these images. The usage of a hemispherical camera, edge detection and feature matching software produce what amounts to Research partially supported by the first author’s NSF RUI grant CCR-0105507. Part of the work was conducted during the Workshop on Rigidity Theory and Scene Analysis, organized by the first author at McGill University Bellairs Institute in Barbados in Jan. 2002, with partial support from NSF grant CCR-0203224. M.T. Goodrich and S.G. Kobourov (Eds.): GD 2002, LNCS 2528, pp. 372–373, 2002. c © Springer-Verlag Berlin Heidelberg 2002 Camera Position Reconstruction and Tight Direction Networks 373 be a 3-dimensional direction network. In [1] standard least-squares techniques are used to infer approximations of the vertex positions. The authors report that the solution is very rarely unique, with the information inferred from the images being too weak to allow correct reconstruction. Our Contribution. It is stated in [1] that the determination of the minimal set of distinct pairs needed for a non-degenerate solution is significantly more complicated and depends entirely on the topology of the adjacency graph. Our contribution is to propose a theoretical solution for what seems to be the bottleneck in the full automation of this system, and perform enough preliminary experimentation to validate the feasibility of taking it to the practical level. We formulate the problem in rigidity theoretical terms. This leads to reasonably efficient and implementable algorithms. We have experimented with algorithms in dimension 2. The real case (dimension 3) is more complicated but still reasonably well understood theoretically. Our implementation was done in Mathematica for quick prototyping. The planar case allows to develop a solid theoretical strategy for accurate testing. These preliminary results indicate that our approach may be practical even in dimension 3. The Combinatorial Rigidity approach. If one could understand which edges must additionally be sampled to ensure that a unique solution is found, then one could send the robot to retake some pictures. This being a costlier process than the off-line processing of images, one would like to do only the work that is absolutely necessary. We are led to the following three problems. 1. Decision: Given a direction network, decide whether it is tight or not. 2. Extension: Given a loose direction network, find a set of edges to be added so that the resulting network becomes tight. 3. Extraction: Given an inconsistent direction network, find a set of edges to be removed to obtain a tight network. These problems fall into the rigidity theory category of parallel redrawings. A good starting point is Whiteley ([4] and [3]). Our implementation relies on an algorithm proposed by Sugihara [2], based on matroidal concepts. In the planar case, the duality between fixed-length and fixed-direction rigidity allowed us to design a test generator using Henneberg constructions for tight graphs. Acknowledgements. We thank Seth Teller for suggesting the problem to us, and to Ruth Haas, Brigitte Servatius, Jack Snoeyink and Walter Whiteley for useful references and enjoyable conversations on these topics.